Provably correct, asymptotically efficient, higher-order reverse-mode automatic differentiation
نویسندگان
چکیده
In this paper, we give a simple and efficient implementation of reverse-mode automatic differentiation, which both extends easily to higher-order functions, has run time memory consumption linear in the original program. addition formal description translation, also describe an algorithm, prove its correctness by means logical relations argument.
منابع مشابه
Higher-order Reverse Automatic Differentiation with emphasis on the third-order
It is commonly assumed that calculating third order information is too expensive for most applications. But we show that the directional derivative of the Hessian (Df(x) · d) can be calculated at a cost proportional to that of a state-of-the-art method for calculating the Hessian matrix. We do this by first presenting a simple procedure for designing high order reverse methods and applying it t...
متن کاملAutomatic Generation of Provably Correct Embedded Systems
With the demand for new and complicated features, embedded systems are becoming more and more difficult to design and verify. Even if the design of a system is verified, how to guarantee the consistency between the design and its implementation remains a big issue. As a solution, we propose a framework that can help a system designer to model his or her embedded system using a high-level modeli...
متن کاملAutomatic Generation of Provably Correct Parallelizing Compilers
We show how parallelizing compilers can be automatically derived from denotational deenitions of programming languages. In our approach, the denotational deenition is expressed using deenite clause grammars (syntax speciication) and Horn Logic or Constraint Logic (semantic speciication). The conditions for executing two or more statements in parallel (e.g., GCD test, Banerjee test, or exact tes...
متن کاملEnabling user-driven Checkpointing strategies in Reverse-mode Automatic Differentiation
Abstract. This paper presents a new functionality of the Automatic Differentiation (AD) Tool tapenade. tapenade generates adjoint codes which are widely used for optimization or inverse problems. Unfortunately, for large applications the adjoint code demands a great deal of memory, because it needs to store a large set of intermediates values. To cope with that problem, tapenade implements a su...
متن کامل"To be recorded" analysis in reverse-mode automatic differentiation
The automatic generation of adjoints of mathematical models that are implemented as computer programs is receiving increased attention in the scientific and engineering communities. Reverse-mode automatic differentiation is of particular interest for large-scale optimization problems. It allows the computation of gradients at a small constant multiple of the cost for evaluating the objective fu...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
ژورنال
عنوان ژورنال: Proceedings of the ACM on programming languages
سال: 2022
ISSN: ['2475-1421']
DOI: https://doi.org/10.1145/3498710